System metrics setup #1041
System metrics setup #1041
Conversation
panavenue
commented
Nov 4, 2025
panavenue
commented
- Setup Metric meter and records
- Add 3 new metrics for initial setup
- connect_latencies
- closed_connection_count
- open_connections
panavenue
force-pushed
the
system_metrics_setup
branch
from
8747506 to
1f835d8
Compare
panavenue
force-pushed
the
system_metrics_setup
branch
from
1f835d8 to
ad4a6e6
Compare
enocom
left a comment
enocom
left a comment
There was a problem hiding this comment.
Glad to see the AlloyDB implementation was helpful. Left a couple of comments.
| disableBuiltInMetrics bool | ||
|
|
||
| // clientOpts are options for all Google Cloud API clients. | ||
| clientOpts []option.ClientOption |
There was a problem hiding this comment.
You don't need this anymore, and can just create a client directly with some improvements in the underlying exporter.
| ApplicationName: d.applicationName, | ||
| Region: inst.Region(), | ||
| ClientRegion: "Client-Region-Testing", // TODO: detect client region | ||
| ComputePlatform: "Compute-Platform-Testing", // TODO: detect compute platform |
There was a problem hiding this comment.
See https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/detectors/gcp which will help here.
| n := c.openConnsCount.Add(1) | ||
| trace.RecordOpenConnections(ctx, int64(n), d.dialerID, cn.String()) | ||
| trace.RecordDialLatency(ctx, icn, d.dialerID, latency) | ||
| mr.RecordOpenConnection(ctx, attrs) |
There was a problem hiding this comment.
There's a bug in the AlloyDB implementation around open connections, but wow I can't find it yet. Beware!
There was a problem hiding this comment.
Ah I think I might know what the potential bug -> The GAUGE metric always have more "open" than "close", because we are sending the metric ONLY on every 60s, and when the application shut down, we never record the "-1" of the open_conn GAUGE metric.
Is this the potential bug? (I was testing on how to go around this, but it's gonna be tough to mitigate this one)
